feat(binary): add git binary patch support by weihanglo · Pull Request #61 · bmwill/diffy

weihanglo · 2026-04-13T02:46:12Z

Blocked on #59. Please review from 58f95d7

Basically a port of weihanglo#33.

BTW, this is probably the very last major feature PR. I might have some other small ones for housekeeping and preparation of the new release.

Add parsing and application of git binary patches,
including both literal and delta formats.

This feature is behind the binary Cargo feature.

The implementation is based on

Specification from https://diffx.org/spec/binary-diffs.html
Behavior observation of Git CLI

Some others highlights:

Bump MSRV to 1.75.0 as zlib-rs requiring that.
Parsing of binary format is always available, only decode/apply operations is behind the binary feature.

* Parse `diff --git` extended headers * split multi-file git diffs at `diff --git` boundaries

Compat test for also `git apply`.

Unlike unidiff, gitdiff produces patches for empty file creations/deletions (`0\t0` in numstat) because they carry `diff --git` + extended headers even without hunks. Binary files (`-\t-\t`) are skipped in gitdiff mode for now.

bmwill#62 optimized it

* Added types representing both literal and delta Git binary patches * Added a parser for the `GIT binary patch` format. This doesn't include the patch application (which will be added in later commits) The implementation is based on * Specification from <https://diffx.org/spec/binary-diffs.html> * Behavior observation of Git CLI

Returns the longest valid UTF-8 prefix of the input. For `str` this is the entire input; for `[u8]` it truncates at the first invalid byte. This will be used at call sites where generic `T: Text` input needs to be narrowed to `&str` for parsers that only handle ASCII data (e.g. binary patch base85 content).

The API was stabilized in 1.73. The lint was added in 1.93. This is required for a MSRV bump to 1.75

This is a preparation for binary diff application support. * Git binary patch is compressed by zlib hence flate2 * zlib-rs (which is the most performant zlib backend) requires MSRV 1.75.0+ hence the bump.

* Add base85 encoder/decoder and Git delta format decoder. * Wire them into `BinaryPatch::apply() and `apply_reverse()` for decoding zlib-compressed, base85-encoded binary payload. These are feature-gated behind the `binary` feature.

Now both tests require `binary` Cargo feature.

weihanglo · 2026-04-17T16:39:33Z

We use our hand-written one. Alternatives:

base85

https://crates.io/crates/base85

Same RFC 1924 alphabet

MPL-2.0

Doesn't have decode_into to avoid allocation

base85rs

https://crates.io/crates/base85rs

Same RFC 1924 alphabet

decode -> Option not good for error reporting

Doesn't have decode_into to avoid allocation

ascii85

https://crates.io/crates/ascii85

It's not Git base85

z85

https://crates.io/crates/z85

ZeroMQ variant

The hand-written one here is solid, and it is nice to avoid an extra dep if we can

bmwill · 2026-04-18T14:32:07Z

+        } else {
+            // ADD instruction
+            let add_len = control as usize;
+            let data = cursor.read_bytes(add_len)?;
+            result.extend_from_slice(data);
+        }


Its been a while since i've implemented the delta compression algorithm but I think that this may be mis handling the case where control == 0x00 which is reserved by by git but here we're treating it as an ADD of 0 bytes.

Although rereading the diffx spec you linked at the top of this file 0x00 doesn't seem to be called out as special or reserved so we could probably go either way.

Good catch! This is definitely worth a compat test. I dare not reading GPL source to confirm that 😨.

bmwill · 2026-04-18T14:37:58Z

+        });
+    }
+
+    let mut result = Vec::with_capacity(header_mod_size as usize);


This can be possibly dangerous as we're reading a u64 value from a possibly malformed set of delta instructions which could lead to some inputs causing an instant OOM. We probably want to try and cap the amount we're willing to allocate up front in order to avoid small adversarial inputs from crashing the process.

bmwill · 2026-04-18T14:41:31Z

+    fn decode_data(binary_data: &BinaryData<'_>) -> Result<Vec<u8>, BinaryPatchParseError> {
+        use std::io::Read;
+
+        let compressed = decode_base85_lines(binary_data.data)?;
+
+        let mut decoder = flate2::read::ZlibDecoder::new(&compressed[..]);
+        let mut decompressed = Vec::new();
+        decoder
+            .read_to_end(&mut decompressed)
+            .map_err(|e| BinaryPatchParseErrorKind::DecompressionFailed(e.to_string()))?;
+
+        Ok(decompressed)
+    }


We don't seem to do any checks that the decompressed data matches the size that is present in BinaryData.size. Do we want to do an extra check for that to ensure the decompressed data is the expected size?

bmwill · 2026-04-18T14:46:44Z

BTW, this is probably the very last major feature PR. I might have some other small ones for housekeeping and preparation of the new release.

Awesome! hopefully getting all this in wasn't too painful.

I have a few smaller things that I think would be nice to add or polish myself so hopefully i can find the time to do that before we are ready to do a release, although I won't hold up a release if i don't get around to those things.

weihanglo · 2026-04-18T17:28:36Z

I have a few smaller things that I think would be nice to add or polish myself so hopefully i can find the time to do that before we are ready to do a release, although I won't hold up a release if i don't get around to those things.

Feel free to dump your thoughts in an issue. I can help clean them up!

Here is a rough list in my mind:

Error handling
- big parse error enum maybe
- precise byte positions or spans
- user have ways to distinguish between different error kinds, at least some ways to distinguish most possible wanted ones.
Public API. We need to review what we expose and if it is in a good shape before release
Performance benchmark?
Of course a changelog updates (and probably some changelog infra via cargo-release, release-plz, cargo-smart-release, or just simple a CI release job triggered by tag)

weihanglo force-pushed the binary-patch branch 5 times, most recently from 61c9c2b to 531e8d9 Compare April 14, 2026 05:00

weihanglo mentioned this pull request Apr 14, 2026

feat(patch_set): PatchSet::parse_bytes for raw byte input #64

Closed

weihanglo marked this pull request as draft April 15, 2026 22:57

weihanglo added 5 commits April 17, 2026 10:18

feat(patch_set): git diff parsing/application

da27abf

* Parse `diff --git` extended headers * split multi-file git diffs at `diff --git` boundaries

test(compat): add git apply compatibility tests

da4cf13

Compat test for also `git apply`.

test(replay): add gitdiff mode support

d4ff209

Unlike unidiff, gitdiff produces patches for empty file creations/deletions (`0\t0` in numstat) because they carry `diff --git` + extended headers even without hunks. Binary files (`-\t-\t`) are skipped in gitdiff mode for now.

chore(ci): run gitdiff also in replay workflow

dfbf385

chore(ci): we can now replay more commits!

065d801

bmwill#62 optimized it

weihanglo force-pushed the binary-patch branch 3 times, most recently from 7101ee1 to a2190d2 Compare April 17, 2026 16:08

weihanglo added 7 commits April 17, 2026 12:28

feat(patch_set): wire binary diff parsing

4e10450

refactor: clippy manual_div_ceil

e58ff79

The API was stabilized in 1.73. The lint was added in 1.93. This is required for a MSRV bump to 1.75

chore: dependency flate2 behind binary feature

899c257

This is a preparation for binary diff application support. * Git binary patch is compressed by zlib hence flate2 * zlib-rs (which is the most performant zlib backend) requires MSRV 1.75.0+ hence the bump.

test: add binary patch to compat and replay tests

bc427f8

Now both tests require `binary` Cargo feature.

weihanglo force-pushed the binary-patch branch from a2190d2 to bc427f8 Compare April 17, 2026 16:30

weihanglo commented Apr 17, 2026

View reviewed changes

weihanglo marked this pull request as ready for review April 17, 2026 16:39

bmwill reviewed Apr 18, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(binary): add git binary patch support#61

feat(binary): add git binary patch support#61
weihanglo wants to merge 12 commits intobmwill:masterfrom
weihanglo:binary-patch

weihanglo commented Apr 13, 2026 •

edited

Loading

Uh oh!

weihanglo Apr 17, 2026

Uh oh!

bmwill Apr 18, 2026

Uh oh!

bmwill Apr 18, 2026

Uh oh!

weihanglo Apr 18, 2026

Uh oh!

bmwill Apr 18, 2026

Uh oh!

bmwill Apr 18, 2026

Uh oh!

bmwill commented Apr 18, 2026 •

edited

Loading

Uh oh!

weihanglo commented Apr 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

weihanglo commented Apr 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

weihanglo Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

bmwill Apr 18, 2026

Choose a reason for hiding this comment

Uh oh!

bmwill Apr 18, 2026

Choose a reason for hiding this comment

Uh oh!

weihanglo Apr 18, 2026

Choose a reason for hiding this comment

Uh oh!

bmwill Apr 18, 2026

Choose a reason for hiding this comment

Uh oh!

bmwill Apr 18, 2026

Choose a reason for hiding this comment

Uh oh!

bmwill commented Apr 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

weihanglo commented Apr 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

weihanglo commented Apr 13, 2026 •

edited

Loading

bmwill commented Apr 18, 2026 •

edited

Loading